1. Field of the Invention
This invention relates generally to computer-implemented discovery of the topology of a multitier compute infrastructure, including those that host multitier applications. In a specific embodiment, it relates to discovery of compute infrastructures that host multitier applications that employ a software component architecture, for example based on the Java 2 Platform, Enterprise Edition (J2EE) or .NET.
2. Description of the Related Art
With advances in computing, networking and software technology, an increasing number of applications are implemented on multitier compute infrastructures. For example, many Internet applications are implemented on a compute infrastructure that can be divided into three tiers: network, compute and application. One advantage of a multitier compute infrastructure is that different tiers can function at different “levels” while still interoperating with each other. In the three-tier example, the network tier operates at the “lowest” level, the compute tier operates on top of that, and the application tier operates at the “highest” level. As a result, enterprise and other applications can be distributed among the tiers in a fashion that results in greater optimization. For example, if a certain functionality is desired, it is not required that the functionality be implemented in a monolithic piece of software installed on a particular computer in a specific location within the network. Rather, the overall functionality can be distributed among different components within the multitier compute infrastructure.
Software component architectures such as Java 2, Enterprise Edition (J2EE) and .NET are one approach which takes advantage of this flexibility. Software functionality is divided among different software components, each of which can run on a different computer located at a different network address. Each of the software components, computers and the network topology may be optimized for efficiency, security, scalability or other factors. For example, in the monolithic approach, a single code base and a single computer may be called upon to handle user requests for enhanced images, retrieve raw images from a large image warehouse and perform complex calculations to enhance the images. With the component approach, one software component and server could handle user requests, another could retrieve images, and a third could perform the calculations. Each of these can be optimized for its specific task and the same components can be used for more than one application. The overall enterprise is also more scalable since incremental capacity can be added by adding more components.
One drawback of the multitier and software component approaches is that, typically, many components are used in order to implement the desired functionality. For example, the software portion can be implemented by a large number of software components, each possibly executing on a different server, with the servers located on different networks. Software components may not even be executing on the same server each time. The real-time execution load can be load balanced among a server farm, for example. Currently, it is not uncommon for an enterprise application to have thousands of moving parts, many of which must work in tandem with each other in order for the overall enterprise application to function properly. In addition, multiple relationships between components exist within each tier, as well as across tiers of the compute infrastructure. For example, in the application tier, a web server and application server might work together to handle user requests. Cross-tier relationships can be more complex, such as those linking the web server, DNS server and access router with each other, but these often are the relationships that have a direct bearing on the availability, performance and security of the overall application.
Due to this increased complexity, managing a multitier compute infrastructure is more difficult. Tasks such as installation, configuration, activation, upgrade and monitoring of overall enterprise functions are more complex compared to a situation where a monolithic piece of code executes on a single computer in a fixed network location. This is aggravated by the fact that the component approach can significantly reduce the development cycle time. It is not uncommon to have a J2EE application undergo ten to twelve upgrades each calendar year, with two of those being major upgrades to underlying functionality. In the meantime, it is increasingly more difficult to install and monitor the upgrades.
Enterprise management capability has not kept pace with the shorter development cycles. For example, the task of deploying an upgrade is largely a manual task, even today. Initially, the application deployment team assembles the various software components making up the enterprise application, manually scans configuration files, and checks them against system documentation and physical network and compute configurations for consistency and completeness. The product of this effort is an inventory that should pinpoint omissions or conflicts prior to staging. However, as the scope of enterprise applications expands and the different tiers become more distributed, the likelihood of uncovering all issues prior to staging decreases. Missed issues are addressed by troubleshooting after deployment. But troubleshooting can be time-consuming as the root causes may depend on complex cross-tier relationships. Not only does this add expense but it can also result in lost revenue as roll out of the enterprise application is delayed. In addition, cross-tier troubleshooting and, more generally, the management of a multitier compute infrastructure are most effectively performed by dedicated teams whose members are skilled in the application, compute and network tiers. It can be difficult to find these people and the IT headcount can be the limiting factor on scaling up an enterprise operation.
Part of the problem is that currently available management tools are mostly limited to a single tier. This is because many of these tools were developed for system administrators who were responsible only for a single tier. That is, one system administrator would be responsible for networking, another for computers, and another for software loaded on the computers. Single-tier tools would give some visibility into the tier for which the system administrator had responsibility, but did not give visibility into cross-tier relationships or interactions. This is problematic since the trend is towards more numerous and more complex cross-tier relationships.
Single-tier tools also do not give direct visibility into the service which is a business' end goal. For example, in the image enhancement example, the business is really interested in the delivery of enhanced images, not in the congestion level of its internal routers or the state of its internal network. The router and network are of interest only to the extent that they impact the business service of delivering enhanced images but, with single-tier tools, it is difficult, if not impossible, to determine this relationship. As a result, the relationship typically must be manually pieced together, one tier at a time, and often using knowledge that resides only in some key employee's head. This is both time-consuming and risky—for example, if the key employee were to leave the company.
Others are attempting to address these shortcomings. Much effort is currently being spent on approaches based on monitoring. OpenView and Tivoli are examples of efforts in this general direction. Management tools can monitor individual components in the infrastructure through instrumentation with increasing detail and sophistication. This can give improved visibility into the individual component but does not effectively address cross-tier visibility or the relationship between a component and a business service. For example, processor throughput, server availability and similar metrics at best can only give indirect visibility into business services, for example whether customers have access to an enterprise application and can perform promised tasks at published service levels.
Another approach focuses on fast and/or automated deployment of application components. Loudcloud and CenterRun are two companies that appear to have efforts in this area. These tools typically automate the deployment of application components. For example, if a patch is to be distributed to 100 instances of an operating system, this tool might automate that process. However, in order to use this tool, someone must know where the 100 instances are located. Furthermore, if the patch itself requires an upgrade in some other piece of software in order to run properly, someone must also remember that. Hence, these tools might reduce the cost and error of manually deploying the patch, but they typically do not increase cross-tier visibility or visibility into business services.
Thus, there is a need for better tools and techniques for managing a multitier compute infrastructure, including those that are implementing a software component architecture.